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ABSTRACT 

In  order  to  automatically  enhance  and  restore  images,  especially  those  taken  from  underwater  environments  where 
scattering  and  absorption  by  the  medium  strongly  influence  the  imaging  results  even  within  short  distances,  it  is  critical 
to  have  access  to  an  objective  measure  of  the  quality  of  images  obtained.  This  contribution  presents  an  approach  to 
measure  the  sharpness  of  an  image  based  on  the  weighted  gray-scale-angle  (GSA)  of  detected  edges.  Images  are  first 
decomposed  by  a  wavelet  transform  to  remove  random  and  part  medium  noises,  to  augment  chances  of  true  edge 
detection.  Sharpness  of  each  edge  is  then  determined  by  regression  to  determine  the  slope  between  gray-scale  values  of 
edge  pixels  versus  locations,  which  is  the  tangent  of  an  angle  based  on  grayscale.  The  overall  sharpness  of  the  image  is 
the  average  of  each  measured  GSAs,  weighted  by  the  ratio  of  the  power  of  the  first  level  decomposition  details,  to  the 
total  power  of  the  image.  Adaptive  determination  of  edge  widths  is  facilitated  by  values  associated  with  image  noise 
variances.  To  further  remove  the  noise  contamination,  edge  widths  less  than  corresponding  noise  variances  or  regression 
requirement  arc  discarded.  Without  losing  generality  while  easily  expandable,  only  horizontal  edge  widths  arc  used  in 
this  study.  Standard  test  images  as  well  as  those  taken  from  field  are  used  to  be  compared  subjectively.  Initial 
restoration  results  from  field  measured  underwater  images  based  on  this  approach  and  weakness  of  the  metric  arc  also 
presented  and  discussed. 

Keywords:  image  quality,  sharpness,  restoration,  underwater,  MCM,  WGSA,  MTF,  wavelet 

I,  BACKGROUND 

Image  quality  has  been  an  interesting  and  important  research  subject  in  digital  image  processing,  especially  with  the 
rapid  expansion  of  digital  cameras,  scanners,  and  printers  into  the  everyday  life  of  most  households  in  this  digital  age. 
Although  the  term  implied  a  rather  broad  description  of  the  quality  of  the  image  obtained,  it  is  important  to  understand 
the  scope  of  the  term  and  the  context  it  is  within.  When  it  comes  to  the  image  quality,  the  most  common  criteria  is  the 
sharpness  of  the  image,  which  represents  the  ability  to  present  details.  This  is  also  referred  to  as  the  resolution,  and  often 
studied  in  terms  of  spatial  (angular)  frequency.  Contrast  of  an  image  is  closely  related  to  the  sharpness  or  resolution  of 
an  image,  and  usually  defined  by  the  differences  between  lighter  and  darker  areas  then  divided  by  the  combined 
brightness  [I,  2].  Artifacts  and  distortions  are  understandably  important  parameters  in  defining  image  quality  as  well, 
especially  in  medical  imaging,  but  often  received  little  attention  in  other  research  areas.  The  level  of  noises  present  in  an 
image  can  be  critical  in  certain  circumstances,  especially  when  the  signal-to-noise  ratio  (SNR)  is  low,  or  for  the  purpose 
of  testing  different  image  compression  algorithms. 

Like  quantifying  the  performance  of  an  imaging  system  with  a  single  number,  using  a  single  value  to  describe  the 
quality  of  an  image  is  at  least  equally  difficult  and  seemingly  impossible,  considering  the  vast  differences  amongst 
imaging  systems,  contents  of  imageries,  and  variations  in  environmental  conditions  which  involve  lighting  and  medium 
effects.  However,  the  importance  and  value  of  a  well-defined  metric  can  be  tremendous,  as  part  of  the  computer 
cognition.  The  ability  of  a  human  being  to  tell  the  difference  of  the  image  quality  in  a  glimpse  is  critical  in  digital  image 
processing,  when  automation  is  desired,  both  for  restoration  of  degraded  imageries  in  post-processing,  as  well  as  real¬ 
time  imaging  enhancement.  Furthermore,  once  the  ability  is  passed  on  to  the  computer,  ie,  a  dependable  metric  is 
defined,  vision-enabled  robots  can  go  beyond  what  human  can  do,  such  as  gaining  independence  which  is  important  in 
harsh  environments,  and  perform  the  same  task  in  much  finer  precisions  and  at  increased  speed. 

Understandably,  the  perceptual  image  quality  is  dependent  of  the  individual  viewer  and  so  is  the  value  of  a  metric,  but  a 
summation  of  many  opinions  often  regress  to  a  common  ground  or  a  mean  opinion  score  (MOS),  which  is  often  used  as 
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a  benchmark  for  testing  the  effectiveness  of  a  metric  [3].  Mathematical  formulas  are  developed  to  quantify  these 
perceptual  quality  metrics  such  as  the  perceptual  distortion  metric  (PDM)  [4},  This  is  generally  referred  to  as  a 
subjective  metric  where  human  intelligence  and  cognition  is  involved,  and  is  often  a  function  of  human  visual  system. 
An  objective  image  quality  metric,  on  the  other  hand,  aims  to  establish  a  measure  of  image  quality  regardless  of  the 
human  vision  system  response.  There  are  two  types  of  objective  approaches.  A  majority  research  has  been  done  in  the 
area  where  one  compares  a  degraded  image  to  an  ideal  version,  or  reference,  to  aid  algorithm  development,  such  as 
those  in  image  compression  for  encoding  and  transmission.  The  common  approaches  in  determining  the  quality  include 
the  use  of  peak  signal- to-noisc  ratio  (PSNR)  and  the  mean  square  errors  (MSE)  [3], 

For  the  type  of  imageries  we  are  interested  in,  such  as  field  measured  undemater  imageries,  a  no-reference,  or  blind, 
objective  image  quality  metric  is  needed,  in  order  to  achieve  these  goals:  1)  to  define  faithfully  the  quality  of  an  image 
without  a  priori  knowledge  in  different  environment  (ie  different  water  optical  properties);  2)  independent  of  the 
content  of  an  image;  3)  immune  or  less  sensitive  to  noises,  especially  those  caused  by  the  multiple  scattering  in  turbid 
underwater  environments.  The  establishment  of  such  metric  is  a  critical  component  in  the  automated  image  restoration, 
where  the  computer  system  needs  to  know  tlwhen*'  to  stop,  and  determine  if  it  has  found  the  best  result,  and  if  the  “best** 
result  is  acceptable  in  a  comparable  underwater  environment. 

Different  objective  image  quality  metrics  exist,  but  to  the  authors*  knowledge,  there  is  no  universal  metric  that  handles 
all  image  types  well  For  example,  quality  associated  with  aerial  imageries  through  the  atmospheric  turbulence  taken  by 
a  reconnaissance  camera  will  be  different  from  those  images  taken  over  a  short  distance  by  a  camera  mounted  on  a  ear 
affected  only  by  motion  blur,  and  while  the  latter  still  different  from  images  taking  in  a  turbid  underwater  environment 
such  as  in  an  estuary  or  harbor  area.  It  is  practical  and  possible,  however,  to  compose  a  quality  metric  for  a  given 
environment  and  task  that  is  well-defined.  One  good  example  is  the  National  Imagery  Intcrprctability  Scale  (N1IRS)  and 
the  associated  general  image  quality  equation  (GIQE)  [5].  NIIRS  is  primarily  a  metric  designed  for  image  quality 
assessment  for  satellite-based  systems  and  as  its  name  implied,  it  focuses  on  the  scale  or  the  ground -samp led  distance 
(GSD).  Briefly,  other  efforts  in  objective  metrics  included  those  that  measure  the  image  quality  by  its  sharpness  using 
the  gradient  or  slope  of  edges  [6],  by  the  perceptual  blur  which  measures  local  blur  values  based  on  all  edge  widths  [7], 
by  a  variance  approach  which  assumes  smoother  edges  correspond  to  less  variance  [8],  by  the  histogram  frequency  that 
is  associated  with  non-zero  transformed  coefficients  [9],  by  the  area  under  MTF  to  separate  high  frequency  and  low 
frequency  contributions  [10],  via  the  power  spectrum  which  measures  the  ratio  of  high  and  low  frequency  energy  to  the 
total  to  reflect  details  of  the  image  studied  [11],  and  by  a  wavelet-based  perceptual  metric  that  applied  a  discrete  dyadic 
wavelet  transform  to  obtain  edge  constraints  [12].  The  relatively  new  wavelet  transform  helps  to  preserve  the  edge 
characteristics,  which  is  especially  useful  in  the  situation  associated  with  discontinuities  [13]. 

Degradation  of  the  image  quality  in  a  scattering  medium  has  been  studied,  where  atmospheric  effects  such  as  turbulence 
and  scattering  by  aerosols  dominate.  These  studies  are  mostly  focused  on  modeling  the  optical  transfer  function  (or 
modulation  transfer  function,  MTF,  when  phase  information  can  be  neglected),  in  an  effort  to  restore  the  images 
obtained,  such  as  in  air  reconnaissance  or  astronomy  studies  [14,1 5].  The  scattering  behavior  is  different  in  the  situation 
of  natural  waters,  where  strong  forward  scattering  dominates.  Anyone  with  experiences  in  coastal  waters,  especially 
those  inside  a  harbor,  or  estuary  areas  like  Mississippi,  has  a  first-hand  look  at  how  visibility  could  quickly  reduce  to 
zero  in  a  matter  of  a  few  feet.  The  same  applies  to  regions  of  strong  re- suspensions  from  the  bottom,  both  in  coastal 
regions  as  well  as  in  the  deep  sea.  The  images  obtained  under  such  conditions  arc  often  severely  degraded  or  blurred. 
The  extent  of  such  blurring  can  be  described  by  the  MTF  of  the  medium  which  includes  water  itself,  constituents  within 
such  as  particulates  (both  organic  and  inorganic)  [1].  Theoretically,  such  effects  can  be  reversed  by  deconvolving  the 
MTF  of  the  medium  from  the  resulting  images  [2].  In  reality,  the  effectiveness  is  often  hindered  by  errors  associated 
with  the  modeling  efforts  such  as  MTF  and  approximations  applied  with  the  small  angle  scattering.  The  noises  from 
field  measurement  results  are  also  part  of  the  mix.  Additionally,  due  to  the  small  incremental  quality  improvements  in 
restoring  badly  degraded  images,  it  is  hard  to  judge  if  one  restored  result  is  better  than  the  other,  which  is  critical  in  an 
automated  process.  For  these  reasons,  it  is  necessary  to  develop  a  method  to  determine  objectively  the  quality  of 
resulting  image  that  has  closer  ties  with  the  environment.  This,  in  turn,  can  be  used  to  better  determine  the  more  specific 
issues  affecting  imaging  in  underwater  environment,  namely,  low  lighting  thus  tow  signal  to  noise  levels,  fluctuations 
caused  by  the  medium,  and  multiple  scattering  contributions. 
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This  approach  improves  upon  previous  efforts  especially  [7,  1 1,  12].  Recent  advance  of  wavelet  research  provides  an 
excellent  tool  for  this  purpose,  as  wavelets  are  multi-resolution  in  nature.  We  use  wavelet  transforms  to  remove  the 
effect  of  scattering  when  locate  edges,  and  further  apply  the  transformed  results  in  restraining  the  perceptual  metric.  The 
medium  MTF  is  preserved  via  ESF  during  measurements.  Edge  profiles  are  determined  by  incorporate  noise  variance 
introduced  by  the  scattering 


2.  IMAGE  QUALITY  METRIC  BASED  ON  THE  WEIGHTED  GRAYSCALE  ANGLE 

Our  approach  is  a  wavelet-decomposed  and  denoised,  perceptual  metric  constrained  by  a  power  spectrum  ratio,  and  is 
depicted  in  Fig.  1,  To  properly  define  the  sharpness  of  an  image,  it  is  critical  to  correctly  determine  the  edge  profile. 
Due  to  the  strong  scattering  of  the  medium,  some  photons  can  be  scattered  into  rather  large  angles  away  from  the 
straight  path,  resulting  in  tow  photons  count  at  the  ends  of  the  edge,  and  can  be  easily  classified  as  “noise".  This  is  in 
fact  the  extension  or  part  of  the  “broadening"  of  the  edge  caused  by  the  scattering,  therefore  preserving  these  extensions 
while  removing  true  noise  is  critical  in  denoising  process.  For  this  reason,  denoising  is  carried  out  by  applying 
Daubechies  wavelets  [16],  to  obtain  more  consistent  result  at  locating  the  edges  of  the  image.  The  edge  profiles  of  the 
image  are  then  found  using  the  Canny  edge  algorithm  [17]. 
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Fig.  \ .  Outline  of  the  WGSA  image  quality  metric  calculation.  See  text  for  details. 


With  the  edges  located,  the  program  explores  to  determine  the  profile  of  each  edge  along  the  horizontal  direction, 
extending  to  the  left  and  right  of  the  edge  pixel  determined  by  the  Canny  algorithm,  which  is  likely  the  center  of  the 
edge.  Considering  the  randomness  of  the  image  contents  and  its  orientation,  using  edge  profiles  only  at  one  direction 
does  not  invalidate  the  effectiveness  of  the  approach.  To  be  computationally  thrifty,  only  one  out  of  every  10  edge  lines 
is  calculated. 

A  typical  edge  profile  from  an  underwater  image  can  be  seen  in  Fig.  2,  which  is  taken  from  an  actual  underwater  image 
frame.  The  edge  pixels  are  determined  by  the  following  criteria  applied  to  the  original  image:  the  continuous  ascending 
(or  descending)  in  one  direction,  in  grayscale  values  with  a  step  size  in  grayscale  (SSG)  should  be  no  less  than  a  pre¬ 
determined  value.  This  pre-determ  ined  value  is  calculated  based  on  the  image  noise  variances  using  an  adaptive  Wiener 
filter  approach  [18].  This  is  especially  important,  as  it  contains  information  related  to  the  medium  scattering 
characteristics  in  order  to  find  the  correct  edge  widths.  The  resulting  profile  is  essentially  the  denoised  edge  spread 
function  (ESF),  or  in  NHRS  terms,  the  relative  edge  response  (RER)  [5],  which  is  directly  related  to  the  MTF  of  the 
medium. 
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Fig.  2.  An  image  line  profile  in  raw  grayscale  values  from  an  underwater  image  frame. 


With  all  the  pixels  in  the  edge  profile  determined  a  simple  linear  regression  is  applied  (Fig.2)  to  determine  the 
grayscale  angle  (GSA)  of  the  profile,  which  determines  the  raw  sharpness  of  the  edge.  When  all  edges  arc  measured,  the 
average  value  gives  the  overall  measure  of  the  sharpness  of  the  image.  Very  short  edges  are  excluded  from  this 
calculation  to  avoid  contaminations  by  in -water  constituents  such  as  rare  large  panicles  or  large  planktons.  An  image  is 
declared  *4edgeless"  or  not  suitable  when  there  are  not  enough  edges  found.  To  properly  address  the  amount  of  noise  (ie 
high  spatial  frequency  components),  and  represent  different  lighting  conditions  in  the  underwater  environment,  it  is 
necessary  to  introduce  a  weighting  factor  that  is  associated  with  the  power  of  the  image  as  a  function  of  details.  It  is 
found  that  this  is  better  accomplished  by  using  the  ratio  between  the  first-level  wavelet  decomposed  details,  to  the  total 
power  of  the  original  image.  Information  on  wavelet  transforms  can  be  found  in  many  references  such  as  [19].  This 
result  is  inline  with  previous  power  spectrum  approach  [l  l]. 

Applying  the  above  algorithm  to  an  image  obtained  in  a  controlled  underwater  environment  {active  source,  no  ambient 
light,  pre-determined  absorption  and  scattering  values)  [20],  the  original  and  the  restored  version  are  shown  in  Figs.  3, 
with  corresponding  WGSA  values  (0  005  and  0. 14  respectively).  As  a  comparison,  two  standard  images  were  used  (Fig. 
4),  one  is  the  original  Lena,  and  the  other  is  a  Gaussian  blurred  version.  Their  WGSA  values  arc  0.60  and  0,15 
respectively.  Notice  that  the  blurred  version  of  Lena  visually  appears  better  than  the  restored  image  in  Fig.  3,  although 
they  have  rather  similar  WGSA  values.  This  is  a  good  example  about  the  uniqueness  amongst  different  imaging 
environments.  The  defined  metric  has  been  used  in  our  automated  image  restoration  program,  and  results  demonstrate 
consistency  over  more  than  30  images  tested  in  different  optical  conditions  and  attenuation  ranges.  In  some  extreme 
conditions  (eg  very  low  contrast),  WGSA  will  give  false  values.  Another  issue  with  the  current  approach  is  the  un-cvcn 
distribution  of  WGSA  values.  This  is  probably  the  result  of  two  factors.  One  has  to  do  with  the  type  of  wavelets  used. 
The  other  is  related  to  the  pre-set  SSG  which  is  associated  with  noise  variances.  Different  wavelet  decomposition 
approaches  can  be  explored,  along  with  analysis  of  ESF,  to  overcome  these  problems. 


3.  CONCLUSION 

To  facilitate  the  goal  to  automatically  restore  degraded  imageries  obtained  from  the  underwater  environment,  an  image 
quality  metric  is  defined  that  is  tuned  to  better  respond  to  the  environmental  parameters.  It  is  based  on  the  wavelet 
denoised  sharpness,  weighted  by  its  normalized  power  spectrum  of  decomposed  wavelet  details.  The  medium  scattering 
effects  are  accounted  for  and  treated  accordingly  in  the  current  approach.  Initial  results  show  that  the  metric  defined 
provides  a  consistent  measure  to  both  the  original  and  improved  imageries,  and  is  suitable  for  automated  restoration 
purposes.  Further  validation  will  be  carried  out  using  a  larger  image  dataset,  such  as  those  similar  to  N1IRS, 
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Fig,  3<  The  original  and  restored  underwater  images.  The  original  (top)  has  WGSA=  0.005  while  the  restored 
has  a  value  of  0, 1 4. 
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Fig.  4.  The  image  quality  of  Lena  and  blurred  version  for  comparison.  WGSA  values  arc  0,60  and  0.15 
respectively. 
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